feat: implement & test `open_at` by Sl1mb0 · Pull Request #363 · influxdata/datafusion-udf-wasm

Sl1mb0 · 2026-02-09T16:38:51Z

Closes #336

I followed the specification described here. Given that we will eventually need to test this against the wasi test-suite, any differences between the aforementioned spec and the test-suite will favor the test-suite; so significant changes may need to be made in the future.

Describe your proposed changes here.

I've read the contributing section of the project CONTRIBUTING.md.
Signed CLA (if not already signed).

host/src/vfs/mod.rs

Copilot

Pull request overview

Implements POSIX/WASI-like open_at behavior in the in-memory VFS so guests can open existing nodes and create/truncate files (a prerequisite for enabling host-side FS writes), and updates unit/integration tests to reflect the new semantics.

Changes:

Refactors VFS path resolution and expands VfsCtxView::open_at to handle CREATE, EXCLUSIVE, DIRECTORY, and TRUNCATE.
Adds extensive unit tests around open_at behavior and updates integration test expectations/snapshots.
Makes Limiter::grow take &self (internally synchronized), simplifying ownership/mutability in component setup.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
host/src/vfs/mod.rs	Implements new `open_at` semantics, factors path traversal helper, and adds unit tests.
host/src/limiter.rs	Changes `grow` to `&self` to allow shared use behind a mutex.
host/src/component.rs	Adjusts limiter initialization to match the new `Limiter::grow` signature/usage.
host/tests/integration_tests/python/runtime/fs.rs	Updates Python FS integration test to exercise create-on-open behavior.
host/tests/integration_tests/evil/fs.rs	Updates large snapshot expectations for new VFS/open behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

host/src/vfs/mod.rs

host/tests/integration_tests/python/runtime/fs.rs

…riptorFlags::Write

Sl1mb0 · 2026-02-17T14:51:49Z

host/src/vfs/mod.rs

+            let limiter = Limiter::new(self.static_limits, &pool);
+            let vfs_state = VfsState::new(limits, limiter);
+            let table = ResourceTable::new();
+            (table, vfs_state)


We return a non-mutable tuple here instead of the VfsCtxView directly, as wrapping these in the VfsCtxView requires that they be mutable references, and rustc doesn't like that 😞

host/src/vfs/mod.rs

crepererum · 2026-02-18T11:39:47Z

host/tests/integration_tests/evil/fs.rs

+    |             | .           | ERR: Is a directory (os error 31)               |
+    |             | ..          | ERR: Is a directory (os error 31)               |
+    |             | /           | ERR: Is a directory (os error 31)               |
+    |             | /bin        | ERR: Bad file descriptor (os error 8)           |


Shouldn't this result in "invalid" or "not found" or something? "Bad file descriptor" kinda sound like an internal mess-up.

There's really only two places we return BadDescriptor. I'm still trying to nail down why exactly this is happening.

Yeah we're hitting this, so the resource we're trying to get doesn't exist in the resource table.

This kind of makes sense though doesn't it? if none of the source/destinations exist; since we start with a new VFS for each test.

The guest code however doesn't create Resource<Descriptor> out of thin air (I think, at least if it's not buggy). So we must have handed out a descriptor that is broken or something? Or do we have some kind of type confusion or broken cast at some point?

Maybe a broken cast? I have no idea what could be the cause. I think if something is going wrong it's related to get_directories - as that's where the resource is initially created and added the ResourceTable. Setting breakpoints here and here, and then running test_copy, we first push a descriptor on to the table, and then when we get it we get BadDescriptor - which is only returned if the the descriptor isn't found in the resource table - so something is not quite adding up.

What's strange is examining the ResourceTable when hitting the BadDescriptor breakpoint shows that it contains an entry; and that entry points to the same value as what's added to it in get_directories.

Reading more about std::fs::copy(from, to), there is this tidbit regarding an error case:

from is neither a regular file nor a symlink to a regular file.

To me this reads that if the file it does try to copy is a directory, it will fail since a directory is neither of those things. And since ., .., and / all exist on the VFS but are directories. If I run this test locally, I do encounter the .. neither a regular file nor a symlink .. error. So that also points to this being the cause of the error.

As for what we should do about this - maybe just remove those values specifically from PATHS? I don't really see that as ideal either, as we definitely want to test those.

I wonder if maybe we should break up the evil tests by files that we expect to be in the VFS vs. files we don't expect to be there?

EG, we could have another constant EXISTING_VFS_PATHS that consists of that list (., .., /) and combine it with PATHS for most tests, but then break it up for certain tests like test_copy. Just kind of a spitball idea ATM though.

…et_node_from_start

host/src/vfs/mod.rs

refactor: require immutable self reference for Limiter::grow

c98f54c

Sl1mb0 force-pushed the tm/open-at branch 12 times, most recently from a5a3acf to 967f281 Compare February 10, 2026 02:07

Sl1mb0 commented Feb 10, 2026

View reviewed changes

host/src/vfs/mod.rs Outdated Show resolved Hide resolved

Sl1mb0 requested review from Copilot and crepererum February 10, 2026 02:15

Copilot started reviewing on behalf of Sl1mb0 February 10, 2026 02:15 View session

Sl1mb0 force-pushed the tm/open-at branch from 967f281 to c389df8 Compare February 10, 2026 02:19

Copilot AI reviewed Feb 10, 2026

View reviewed changes

host/src/vfs/mod.rs Outdated Show resolved Hide resolved

host/src/vfs/mod.rs Outdated Show resolved Hide resolved

host/src/vfs/mod.rs Outdated Show resolved Hide resolved

host/tests/integration_tests/python/runtime/fs.rs Show resolved Hide resolved

Sl1mb0 force-pushed the tm/open-at branch 9 times, most recently from d7bd5c4 to adb81f9 Compare February 10, 2026 04:57

feat: implement & test open_at

23d3040

Sl1mb0 force-pushed the tm/open-at branch from adb81f9 to 23d3040 Compare February 10, 2026 13:36

Sl1mb0 added 2 commits February 17, 2026 06:21

fix: snapshot assertions

35228e7

fix: return ErrorCode::IsDirectory if a directory is opened with Desc…

d211efd

…riptorFlags::Write

Sl1mb0 commented Feb 17, 2026

View reviewed changes

Sl1mb0 force-pushed the tm/open-at branch 3 times, most recently from 6278d31 to 9d61c8c Compare February 17, 2026 15:42

fix: test_copy

f055ee4

Sl1mb0 force-pushed the tm/open-at branch from 9d61c8c to f055ee4 Compare February 17, 2026 15:57

Sl1mb0 requested a review from crepererum February 17, 2026 16:06

crepererum reviewed Feb 18, 2026

View reviewed changes

feat: add shrink() method to Limiter

ce5918c

Sl1mb0 force-pushed the tm/open-at branch 3 times, most recently from 862b2a5 to 259bdb4 Compare February 18, 2026 22:30

refactor: return Result<Option<SharedVfsNode>> from get_VfsCtxView::g…

4f3e854

…et_node_from_start

Sl1mb0 force-pushed the tm/open-at branch from 259bdb4 to 4f3e854 Compare February 18, 2026 22:41

fix: tests

84fd9ca

Sl1mb0 force-pushed the tm/open-at branch 7 times, most recently from 71e926e to c750feb Compare February 19, 2026 15:42

fix: tests

1ef622b

Sl1mb0 force-pushed the tm/open-at branch from c750feb to 1ef622b Compare February 19, 2026 15:56

Sl1mb0 requested a review from crepererum February 19, 2026 17:15

crepererum reviewed Feb 20, 2026

View reviewed changes

host/src/vfs/mod.rs Outdated Show resolved Hide resolved

refactor: use map_err to make match statement more readable

efcb604

Sl1mb0 requested a review from crepererum February 20, 2026 16:13

Comments

Conversation

Sl1mb0 commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Sl1mb0 Feb 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

crepererum Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

Sl1mb0 Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Sl1mb0 Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

crepererum Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Sl1mb0 Feb 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Sl1mb0 commented Feb 9, 2026 •

edited

Loading

Sl1mb0 Feb 19, 2026 •

edited

Loading

Sl1mb0 Feb 19, 2026 •

edited

Loading

Sl1mb0 Feb 20, 2026 •

edited

Loading